1 Experimental Data Analysis

The experiment utilized OpenSim musculoskeletal modeling and deep learning technologies to generate synthetic proprioceptive inputs, simulating the movement process in animals. Key results include the performance comparison of different neural network models, such as TCNs and LSTMs, in predicting neural activity, as well as the performance differences of these models under active and passive movement conditions.

The Results showed (Figure 1) that a normative framework for studying the neural coding of proprioception. By utilizing synthetic muscle spindle inputs and musculoskeletal modeling, neural networks were optimized to solve 16 computational tasks, testing various hypotheses based on learned representations. The study evaluated which type of hypothesis could better explain the neural activity in the cuneate nucleus (CN) and primary somatosensory cortex (S1, area 2) of non-human primates during active and passive center-out reaching tasks. The model types compared include a baseline linear encoding model that directly predicts neural data from experimental data, and three types of neural network models: data-driven models trained end-to-end on experimental data, untrained neural network initializations tested directly on experimental data, and task-driven models trained on synthetic data to perform computational tasks.

It can be seen that (Figure 2) by enhancing 2D character trajectories and projecting them into 3D space, synthetic proprioceptive inputs are simulated using a two-link four-degree-of-freedom (DoF) arm model. Muscle lengths and velocities are calculated from the 3D trajectories using inverse kinematics and musculoskeletal modeling. The figure shows the distribution of joint angles in the behavioral data (above) and synthetic data (below), with the motion statistics of the synthetic dataset designed to encompass the biological movements of non-human primates during center-out reaching. Sixteen computational tasks were designed, reflecting hypotheses about proprioceptive processing, each containing one or several learning objectives. Different neural network architectures were designed to integrate proprioceptive signals in various ways, including spatial-temporal, temporal-spatial, spatiotemporal TCNs, and spatial-LSTM.

Based on the center-out reaching experiments and linear encoding models of neural activity in Figure 3, it can be observed that the top part of the figure displays the total number of neurons for each non-human primate (NHP), while the bottom shows the total number of active (hollow bars) and passive (solid bars) trials. The screen cursor coordinates in the example session represent the end-effector trajectories during the center-out reaching, with colors indicating the target direction. The normalized firing rates of neurons in both active and passive trials are displayed in raster plots, sorted by movement direction. A pairwise comparison of the average firing rates of individual neurons under active and passive conditions was conducted for NHP S (CN) and NHP H (S1). The distribution of single-neuron test explained variance for the baseline linear encoding models was analyzed using task-related and behavioral variables, in both single-variable and multi-variable models.

Figure 4 illustrates the task-driven neural network models predicting neural dynamics during active and passive movements. Time-varying neural predictions for active (top) and passive (bottom) movements were tested on sample units in CN and S1, showing a comparison between experimental spike firing rates (black lines) and predictions (colored lines) based on the best-predicting layer of a 12-layer spatial-temporal neural network model. An example distribution analysis of single-neuron explainability across all NHPs and types of movements was conducted. Statistical significance was calculated using the Wilcoxon signed-rank test. The explained variance between well-trained models and linear models was compared, highlighting that task-driven models provide higher neural explainability compared to data-driven models, especially in active movements.

The study also compared hypotheses based on kinematic tasks in terms of developing features closest to the brain (Figure 5). By comparing the top 3% of models in all computational tasks, Figure 5A shows the distribution of neural explainability for NHP S (CN) and NHP H (S1), while contrasting the predictions of randomly initialized untrained models and linear models. Figure 5B used UMAP embedding space to visualize the difference in neural explainability between task-driven and untrained networks, with each data point representing a group of networks trained for a given computational task. Figure 5C compared the explained variance gains of all neural network models trained on the hand position and velocity task against linear models and the corresponding untrained models. For passive predictions, 5D, 5E, and 5F are similar to the above, showing the analysis results under passive conditions. These results highlight the superiority of task-driven models in predicting neural dynamics compared to linear and untrained models.

Figure 6 indicates a significant correlation between neural network task performance and the model's neural explainability. By analyzing the 10-layer task-driven model, the distribution of the best-predicting layer for each neuron is shown, with different colors representing different NHPs (Figure 6A). Figure 6C reveals the relationship between explained variance and task performance (mean squared error, MSE) for all TCN models (N = 300) trained on the hand position and velocity (HP and HV) tasks, displaying a negative correlation, meaning lower MSE indicates better performance. Figures 6E and 6F show the relationship between explained variance and task performance for trained and untrained models during active and passive movements, respectively, pointing out that trained models have better predictive capabilities than untrained models. Figures 6G and 6H are linear fits for all computational tasks, showing the performance of NHP S during active movement, with error metrics Z-scored for task comparison, where higher values indicate poorer performance.

The study investigated the differences in task-driven neural predictions between active and passive movements, suggesting the potential for top-down modulation at different levels of the proprioceptive pathway (Figure 7). The results showed significant differences in neural activity between active and passive movements during proprioceptive signal processing, indicating that neurons at various levels of the proprioceptive pathway might be regulated by task-related higher cognitive processes. These differences reflect the influence of cognitive and motor control factors not only in the sensory input itself but also in its processing, thus supporting the notion of top-down modulation in complex neural pathways.

2 Analysis of Research Findings

The analysis of the research results indicates that task-driven models have a superior ability to predict neural activity under active conditions compared to passive ones. This finding suggests that the model is more accurate in processing complex neural signals generated during active movements, and this accuracy is not dependent on the number of trials or their duration, indicating good generalization capability of the model. Comparing task-driven models with data-driven models further revealed the superiority of the former in capturing and explaining neural activities. Task-driven models are better at elucidating how motor control and sensory input are integrated and processed in the brain, whereas data-driven models show limitations in explaining complex neural activities. These comparative results emphasize the importance of adopting task-driven approaches in understanding neural encoding and processing mechanisms in neuroscience research.

3 Evaluation of the Research

The results of the research demonstrates the significant advantages of the task-driven modeling approach in understanding and predicting the neural dynamics of proprioception. This method, combined with large-scale synthetic datasets, effectively simulates the motor and sensory processing mechanisms of organisms, providing a powerful tool for in-depth analysis of the functioning of the neural system. The study not only proves the effectiveness of deep learning technologies in processing and interpreting complex neural data but also reveals the subtle dynamics in the proprioceptive process through musculoskeletal modeling. This integrated approach underscores the critical role of computational models in decoding the functions of the neural system, particularly in simulating neural activities and proprioceptive responses. Moreover, this study's methodology offers a robust framework for future exploration of unresolved mysteries in neuroscience, especially in understanding how complex neural networks respond to different types of sensory inputs and motor control tasks.

4 Conclusions

This study explicitly demonstrates that task-driven neural network models can effectively predict the neural dynamics of proprioception in primates. With their exceptional performance under active movement conditions, these models highlight their potent ability to simulate and understand the mechanisms of proprioception. Through an in-depth analysis of various types of task-driven models, the research reveals how these models more accurately capture and explain the complex dynamics of the neural system, especially when processing neural signals generated during active movements. Moreover, comparing the performance of different models emphasizes the importance of the task-driven approach in precisely predicting neural activity and indicates that this method can provide profound insights into how the brain integrates sensory input and motor control. Overall, this study offers valuable methodologies and theoretical foundations for further exploration in the field of neuroscience using task-driven models, particularly in deciphering the processes of proprioception and neural encoding mechanisms.

5 Access the Full Text

Vargas A.M., Bisi A., Chiappa A.S., Versteeg C., Miller L.E., and Mathis A., 2024, Task-driven neural network models predict neural dynamics of proprioception, Cell, 187(7): 1745-1761

https://doi.org/10.1016/j.cell.2024.02.036

Acknowledgement

I sincerely thank Cell for providing free access to papers for a wide audience. The journal's open access policy has enabled me to download, study, comment on, and share this outstanding research paper without any barriers. Cell's initiative has not only greatly promoted the widespread dissemination of scientific knowledge but also provided valuable learning resources for researchers, students, and science enthusiasts worldwide. The spirit of open sharing has profound significance in advancing scientific progress and enhancing the public's awareness and interest in scientific research. I again express my heartfelt gratitude for contribution and support of Cell and look forward to continuing to explore the mysteries of the scientific field through this platform.